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-- The MAILING DATE of this communication appears on the c ver sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH{S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimunn of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )S Responsive to cominunication(s) filed on 18 February 2000 . 
2a)n This action is FINAL. 2b)^ This action is non-final. 

3) D Since this application is in condition for allowance except for fornnal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1-37 \slBre pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) ^ Claim(s) 1-37 is/are reiected. 

Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) \Z\ The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 18 February 2000 is/are: a)n accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

1 1) 0 The proposed drawing correction filed on is: a)n approved b)n disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) n The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)nAI! b)n Some*c)n None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 

Attachment(s) 

1) S Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No{s). . 



2) □ Notice of Drattsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Infomnal Patent Application (PTO-152) 

3) D Infomriation Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 
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DETAILED ACTION 



1. 



This action is in response to the application filed 2/18/2000. 



2. 



Claims 1-37 have been examined and are pending in the application. 



Drawings 



3. The drawings are objected to because: they contain hand-written lines, 
numbers and letters. A proposed drawing correction or corrected drawings are required 
in reply to the Office action to avoid abandonment of the application. The objection to 
the drawings will not be held in abeyance. 



The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the 
United States before the invention thereof by the applicant for patent, or on an international application 
by another who has fulfilled the requirements of paragraphs (1 ), (2), and (4) of section 371 (c) of this 
title before the invention thereof by the applicant for patent. 

4. Claims 1-9 and 1 1-37 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Toomey U.S Patent No. 6,119,147. 

As to claim 1 , Toomey discloses a multi-modal application with a first and a 
second mode processes (multiple 120s, Fig. 2); a multi-modal shell (230, Fig. 2) for 
managing and synchronizing information exchanges between the processes (meet and 



Claim Rejections - 35 USC § 102 
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interact, line 50 colunnn 1); an API (210, Fig. 2) that allows the processes to register 
their commands and actions. 

As to claim 2, Toomey further discloses a registry with a registration table (220, 
Fig. 2) comprises a list of each of the registered commands and corresponding actions 
(lines 38-48 column 6). 

As to claim 3, Toomey further discloses a first and a second mono-mode 
applications (a set of participants, line 48 column 1 ); the shell manages and 
synchronizes information exchanges between these applications (meet and interact, line 
50 column 1). 

As to claim 4, note the discussion of claim 3 above. Toomey further discloses a 
first (first 120, Fig. 2) and a second user interface modality (second 120, Fig. 2). 

As to claim 5, Toomey further discloses a network (130, Fig. 2), and the API is 
implemented using distributed APIs or protocols (lines 3-11 column 2). 

As to claim 6, Toomey further discloses a mechanism (Fig. 1) for converting a 
mono-mode application (120, Fig. 1) to a multi-modal application (105, Fig. 1). 

As to claim 7, Toomey further discloses the mono-mode application is a GUI 
application (user's interface, lines 56-57 column 2); the mechanism provides speech 
enablement of the GUI application by registering the active commands (commands that 
update each user's interface, lines 56-57 column 2) and building a grammar for the 
commands (lines 53-60 column 2). 

As to claim 8, note the discussion of claim 6 above. 

As to claim 9, note the discussions of claims 1 and 7 above. 
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As to claim 1 1 , note the discussion of claim 5 above. 

As to claim 12, Toomey further discloses a multi-modal browser (browser, line 12 
column 8), the first mode process renders a first modality and the second mode process 
renders a second modality (lines 12-20 column 8). 

As to claim 13, Toomey further discloses the first modality is GUI (text 
discussion, line 60 column 1) and the second modality is speech (audio commands, 
lines 60-61 column 1). 

As to claim 14, Toomey further discloses the shell processes a CML file to send 
information in the CML file to the browsers (line 55 column 1 to line 2 column 2). 

As to claim 1 5, Toomey further discloses the CML file encapsulates the 
information in a single modality-independent representation (captured for future replay 
and augmentation in a multi-modal document, lines 58-59 column 1). 

As to claim 16, Toomey further discloses a combination of declarative markup 
languages (text discussion, audio commands, graphics, and documents, lines 60-61 
column 1), 

As to claim 17, Toomey further discloses a single file combining the declarative 
markup languages and synchronization elements to provide tight synchronization 
between the declarative markup languages (interactions are inserted into the multi- 
modal document at a point that is chronological in the meeting to create a synchronous 
meeting, lines 11-13 column 15). 
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As to claim 18, note the discussion of claim 17 above. Toomey further discloses 
a separate file for each of the declarative markup languages (multiple tracks in the 
multi-modal document, lines 59-60 column 1). 

As to claim 19, note the discussions of claims 1-2 above. Toomey further 
discloses updating application states or device states (update user's interface, lines 56- 
57 column 2; update the state, line 55 column 9). 

As to claim 20, note the discussions of claims 2 and 6-7 above. 

As to claim 21 , note the discussion of claim 2 above. 

As to claim 22, note the discussion of claim 1-2 above. 

As to claim 23, Toomey further discloses looking up the command in the table 
and executing the actions on the first and second modalities (lines 18-45 column 7). 

As to claims 24-25, Toomey further discloses registering and executing a 
callback handle for the registered commands (lines 19-44 column 10). 

As to claim 26, Toomey further discloses executing first thread associated with 
the command and triggering second thread to initiate the action (Iine45 column 1 1 to 
line 4 column 12). 

As to claim 27, Toomey further discloses the threads are applets (graphics, line 
61 column 1). 

As to claim 28, Toomey further discloses the threads communicate via socket 
connections (Fig. 2). 

As to claims 29-31 , note the discussions of claims 19-21 above, respectively. 
As to claims 32-37, note the discussions of claims 23-28 above, respectively. 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Toomey. 

As to claim 10, Toomey teaches that his system is implemented on a general 
purpose computer (line 33 column 5). toomey does not explicitly disclose an operating 
system. "Official Notice" is taken that both the concept and advantage of providing for 
an operating system is well known and expected in the art. It would have been obvious 
to include an operating system into the system of Toomey because it would provide the 
interface for meeting sessions between the participants. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to The Thanh Ho whose telephone number is 703-306- 
5540. A voice mail service is also available for this number. The examiner can 
normally be reached on Monday - Friday, 8:30 am - 5:00 pm. 
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Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is 703-305- 
3900. 

Any response to this action should be mailed to: 

Commissioner for Patents 

P.O Box 1450 

Alexandria, VA 22313-1450 
Or fax to: 

• AFTER-FINAL faxes must be signed and sent to (703) 746 - 7238 

• OFFICAL faxes must be signed and sent to (703) 746 - 7239 

• NON OFFICAL faxes should not be signed, please send to (703) 746 - 7240 



TTH 

July 28, 2003 




